Why Accuracy is Not Enough: The Need for Consistency in Object Detection

Object detectors are vital to many modern computer vision applications. However, even state-of-the-art object not perfect. On two images that look similar human eyes, the same detector can make different predictions because of small image distortions like camera sensor noise and lighting changes. This problem is called inconsistency. Existing accuracy metrics do properly account for inconsistency, work in this area only targets improvements on artificial distortions. Therefore, we propose a method use nonartificial video frames measure detection consistency over time, across frames. Using method, show ranges from 83.2% 97.1% datasets multiple tracking challenge. We conclude by showing applying distortion corrections such as WEBP Image Compression Unsharp Masking improve much 5.1%, with no loss accuracy.